Jigglypuff part 1
===

### General Questions ###


investigator: Okay, recording in progress. So, now we can start with the first question: which tasks do you usually use at digital library for? Please tick all answers, which apply and complete your own tasks. Please give short order examples of the tasks you're ticking. So for example, you could say "I'm ticking person search because I like to keep track of myself".


##### GQ1 #####


So, what are the tasks usually use a digital library for? 

Jigglypuff: Okay. So, I usually do not use it for person search, so I will skip this. I use it for paper search. I think you would call it yeah, known item search. So, usually, I only remember some parts of the title and then I would use these as queries, these words, and look for the corresponding paper.

I also use it for venue search to have an overview on recent publications that were submitted to a journal or a conference, for example, I very often use it for BibTeX data. So, I usually get all of the big BibTeX data from dblp because it's nicely formatted and it's one of my primary use cases.

Yes, of course. I also use it to get full text papers. So, I think that's my usual way of how to download full text papers. And yeah, so regarding, I think I would not tick the last. 

investigator: Okay. Is there any other task which you also do which has not been mentioned here? 

Jigglypuff: I don't... I think I don't come up with any good use case right now. 


##### GQ2 #####


investigator: Okay, perfect. Then we can continue to the next question. It's which system or digital library you usually use to solve these tasks? Please tick all answers, which apply and name others, which also apply, but have not been given here. Please give short oral descriptions, why you like, or why you use the specific system? 

Jigglypuff: Okay. So, I usually use the ACM digital library because this holds some major conferences of my research field.

So, for example, on the SIGIR proceedings and I yeah, as mentioned before I also use dblp. Of course, Google scholar, because this is very comprehensive and I use it to look up, yeah, studies or publications from other research fields. Yeah, I sometimes use Springer link, but not that often. Maybe I should tick this. 

investigator: Yeah.

Jigglypuff: Yes. Yeah. And, of course, I also use Google for looking at papers if I'm not successful with all the other digital libraries and yeah, I have two other digital libraries that I have used in the last time. The first one is IR anthology. Yeah.

investigator: Okay. Why do you like this?

Jigglypuff: Because it's domain specific. So, this is specifically related to information retrieval research and yeah, and in this regard it's very, it features all of the venues which are important for retrieval research and in order to avoid paywalls I use the internet archive scholar service. This is, I think, a very new service, which is in a beta phase, so I add this here.

investigator: Okay, nice. So, then we can continue to the next page, if this was all of the systems and digital libraries? Yeah? 

Jigglypuff: Yep. 


### TASK 1 ###


investigator: Okay. Then you should see task one.

Jigglypuff: Okay. Find okay. My task... 

investigator: Yeah. Okay. Consider a following task: find two experts on the topic of your liking. Example topics could be "domain specific query languages" or "hashing functions", but should be from the broader area of computer and information science. You can pick whatever topic you like.

It doesn't have to be one of the two that are given as examples. And yeah, you do not really have to do this task right now, but we are going to talk about how you would do this task. 


##### TASK 1.1 #####


So, the first question is what is your chosen topic?

Jigglypuff: Yeah. So, can I choose my, my research topic of my, dissertation project?

investigator: Yeah, of course!

Jigglypuff: Okay. Then it would be so "reproducible, computational sciences", maybe a little bit more generic. 

investigator: Okay, nice. 

Jigglypuff: Just "reproducible information retrieval". And yeah, the other one would be maybe "online user experiments"?

investigator: Okay. You don't... you only have to use one of them. 

Jigglypuff: Oh, okay. Just one topic. Okay.

investigator: Yeah. 

Jigglypuff: So, yeah, let's pick the "reproducibility" topic. Yeah. 


##### TASK 1.2 #####


investigator: Okay, perfect. And how familiar are you with this topic? 

Jigglypuff: Yeah, so I would not call myself an expert, so but also not a beginner and somewhere in between, maybe in intermediate. 


##### TASK 1.3 #####


investigator: Okay. Nice. And how would you define an expert? 

Jigglypuff: Yeah, this is a good question. Yeah, so maybe an expert in this field would have some publications and would also be a professor. Yeah, or has an academic degree. 

investigator: Okay. Yeah. If it is all...?

Jigglypuff: Yeah. So, for now, maybe...

investigator: That's probably fine. Okay. Yeah. If this is what you call an expert out of the blue, then that's fine. 

Jigglypuff: Okay. 


##### TASK 1.4 #####


investigator: Then we can continue with the next task, if you like with the next question. And this question asks you, how would you solve this task of finding two experts on the topic of reproducibility?

Jigglypuff: Yeah. So, I would rather search for the topic related terms. So, for example, like reproducibility applicability, or repeatability, something like this. And not particularly for an expert person maybe later on if I have found some, several works by the same authors, I would also look up these names. Yeah. 

investigator: How would you search for terms? Where would you search for terms?

Jigglypuff: For, if I'm lucky I have one first publication and which I can yeah, read something about the topic and pick up some of the terminology and look for these terms. Yeah. 

investigator: Okay. But you can consider your specific topic and your familiarity with it, so, ... 

Jigglypuff: Ah, okay. So, it doesn't have to be completely...

investigator: No, no, no, no. 

Jigglypuff: Okay. Yeah, so then I would directly go into the reference section and would get it in order to get an overview of what is cited in this publication. And yeah, I would, based on what is in the references, I would judge if this publication falls into my research field, or if there are many new publications from other sub related fields and based on this, I would judge, if this is the right publication or not, and maybe if it is something in between, I would look up some of the studies that I do not know. Yeah. 

investigator: Okay. So how do you pick the first paper to check the references?

Jigglypuff: Yeah, this is... so I do not want to say that is basically random just by... yeah, or maybe is it random? So, I'm using one of the digital libraries to look up for the keywords and maybe I would pick the let's say first three publications that show up and based on these, I would start my literature research.

investigator: Okay, nice. So then for judging the references, if they are on the topic, do you only check their titles or do you also check like something else and yeah. How do you do that?

Jigglypuff: Yeah, so usually I do it by reading. So beside..., so normally the title is the basis for finding these studies or the publications. Because most of the services I use only use the title. Yes. But once I've found something that seems to fit, I usually judge it on the basis of the abstract and yeah, maybe if the abstract is fine, I will have a look at the conclusions. 

investigator: Okay. So, you do not check the reference section in the PDF of paper, but rather in a website of the first paper, let's say?

Jigglypuff: Yeah. So, if some digital libraries offer this kind of service. And if it is... so for example, I know it from the ACM digital library that they include the references, I think also Springer Link and Yeah. So after having read the abstract, I would jump to the reference section here.

investigator: Okay, nice. And then that, you know, which papers are in a topic and know who might have authored them, what is your next step to finding experts?

Jigglypuff: So maybe it's not that clear me what you would usually do, but maybe I would enter the name of the experts into the search interface. 

investigator: Okay. So the experts or potential experts are the persons who authored those papers that you found relevant from the reference?

Jigglypuff: Yeah, I guess so. Yeah. 

investigator: Okay. 

Jigglypuff: So, let's say I have three or more core papers out of which I extract some of the references and maybe there are intersections. Maybe they cite the same references. In this case the authors of these refers should be experts in these fields. Yeah. 

investigator: Okay. Do you consider all of the authors experts or, I dunno, do you recheck or only pick like a specific person from the author list? 

Jigglypuff: Yeah. This is a good question. So, to be honest I would not consider... so I... in order to say that someone is an expert, I also look for other yeah, sources about the author. So that, assume that the first author is a PhD student. I think we can definitely attribute a lot of work to this person. But I would not consider this as an expert. So, to me, an expert is someone who has maybe some years of knowledge within a broader research field and has found a particular subtopic in this field.

And very often, these kinds of persons are rather one of the last authors in the order. So, let's assume that it's not alphabetically ordered, but if I don't know these persons, I would look for them and maybe have a look at their university pages. If they have one in order to judge, if they can considered it as an expert.

investigator: Okay. Nice. and then the more senior authors you do consider experts. Yeah?

Jigglypuff: Yes. Yeah. 

investigator: Okay. Do you want to add anything else to this process or is that basically it? 

Jigglypuff: I think that that's good for now. Yeah. 


### TASK 2 ###


investigator: Okay. Perfect. Then we can continue to the second task on the next page. 

Jigglypuff: Yeah. Consider task...

investigator: oh, sorry. Consider the following task: find relevant papers from a topic of your liking, which appeared after 2017. Example topics could be "paper recommendation" or "author disambiguation", but should be from a broader area of computer and information science. You can pick the topic you've chosen before, so reproducibility, or you can pick another one if you'd like.


##### TASK 2.1 #####


So, my first question is what is your chosen topic? 

Jigglypuff: Yeah, so I will, once again, chose the reproducibility topic. 


##### TASK 2.2 #####
##### TASK 2.3 #####


investigator: Perfect. So, we can skip question two of the familiarity and now I would want you to define what is relevancy for you?

Jigglypuff: Yeah, so maybe one very simple criterium to judge if a study or publication is relevant is the venue. So, if it is published in a highly ranked journal or conference, I automatically assume that this has some kind of relevance. 

investigator: Okay. 

Jigglypuff: But apart from that, I also think that topical fit is one of the most important criteria. 

investigator: Okay. Do you wanna add anything else to this or is this good and we can continue with the next question?

Jigglypuff: We can, can continue. Yeah. 


##### TASK 2.4 #####


investigator: Okay, nice. So how would you solve the task? How would you find relevant papers on reproducibility, which appeared after 2017? 

Jigglypuff: Yeah. So, if the search interface offers such a feature, I would select all of the publications that appeared after 2017, maybe by adding it to the query or by yes, clicking one option in the interface.

investigator: Okay, so which search interface would you use? 

Jigglypuff: Yeah, for tasks like this, I would usually use dblp.

investigator: Okay, nice. And what would be your query? 

Jigglypuff: Okay, so right now, I don't know how I have to formulate the query. 

investigator: No, no, no, just some general, what would you put into the query? Would it be reproducibility, or would it be with additional terms, or would you refine the query or... yeah?

Jigglypuff: Okay. If we stick to the example of reproducability I would combine it with several related terms. Say by the example of reproducability this would be applicability, generalizability, or also validity. So, terms like this. And yeah, then I would add the year related tier to the query. 

investigator: Okay, nice. So, you would extend the query with known related terms. Would you put them all into one query or would you refine a query step by step and exchange those parts? And what would you do with the results of the query?

Jigglypuff: So, this, of course it is a valid strategy to do it step by step. But usually, I use all of them in one query. 

investigator: Okay. 

Jigglypuff: And then I browse through the result list. It's rather recall oriented. 

investigator: Okay, nice. So, then you browse through the result list and what do you pick as a relevant paper for yourself? Or how do you choose if the paper is relevant? 

Jigglypuff: I think that mostly depends on how the results are displayed. But, very often, it's just, I can just decide it based on the title of the publication and then it somehow should meet my expectations of the topic. Yeah. No, I think expectation's not the right word, but somehow my... I don't know the right word for it right now.

investigator: Do you know the German word? You can also say the German word and we can try to translate it. 

Jigglypuff: Yeah, "meine Vorstellung". 

investigator: Oh, okay. Your, yeah, it is kind of expectations, but yeah, let me recheck. Yeah. Your notion, concept, perception. 

Jigglypuff: Yeah, my conceptual understanding, so if it fits together with my conceptual understanding of the topic then I will click on it and have a more detailed look at it. 

investigator: Okay. And your more detailed look is then to determine if it is relevant or is this like, okay, now you're checking it because it is relevant?

Jigglypuff: No, this would still be to check if this is relevant at all. Because sometimes a title can be promising, but totally misleading. 

investigator: Okay. So then how would your more in-depth check look like? 

Jigglypuff: Okay. So, as I said before, I would check the references and assuming that I've seen some authors before, that publish something in this area of research, I would look for these authors and see if it is somehow related to this network. 

investigator: Where do you get to abstract from? You said you would usually use dblp and yeah. Where would you...? 

Jigglypuff: Yeah, so if I follow the link by dblp, then I would be... yeah, so most of the publications I read or cite are from the ACM digital library. There I'm provided with an abstract even if I don't have access to the full paper. But I think most of the other journals, so I know that Elsevier also shows the abstract and yeah, to be honest, I don't know which service, or which digital library does not display the abstract except for dblp. 

investigator: Okay, nice. So, then you're checking the abstract and you are checking the authors in parallel, or do we do it sequentially or...?

Jigglypuff: Yeah. So, I do it sequentially. Yeah. So, starting with the first author and then yeah, I continue with step by step. Yeah. 

investigator: Okay. Nice. And then with this information, you fetch the most relevant papers, or do you fetch all of them, which are okay-ish? 

Jigglypuff: I would say... so I would decide for the latter. So, assuming that I just have started to get into the topic. I would also keep the okay-ish paper in order to gather more references. So even if the study itself lacks some kind of topical fit, it is very likely that in the references are some more relevant studies. So, there are some more relevant studies. Yeah. 

investigator: So, you would also check the references of the papers to gather more candidates?

Jigglypuff: Yeah.

investigator: Okay. 

Jigglypuff: Also, yeah. I would also check or let's simplify it. So, I would also check the references of the okay-ish paper and see if there are overlaps in the references to have some kind of additional indicator.

investigator: Okay. Nice. Do you want to add anything else to this process? 

Jigglypuff: No, not right now. No. 


### Thank you ###


investigator: Okay. Then I will stop the recording. 

Jigglypuff: Okay. That's it. Okay.

